PR: Add CI workflow for package release on tag push #195

goanpeca · 2020-10-15T00:35:28Z

Simplify release process with package publishing on CI
Adds a check-manifest step
Uses bump2version (the maintained fork)

goanpeca · 2020-10-15T01:09:05Z

@MSeal, still a WIP, but this simplifies a bit the release procedure so that the artifacts can be released on CI based on a tag.

I suggest we use tags that start with v so that we can trigger this with v* in case there are other different tags in the future?

The process would be now:

# Updates to release version and creates a commint and a tag
bump2version release --tag
# Updates to patch dev version and creates a commit
bump2version patch

# Pushes things
git push upstream master
# This will trigger a CI workflow to publish packages
git push upstream --tags

(Assuming upstream poitns to this repo and not a fork, we can update this)

This would make two commits from dev to relase and from release no next patch dev version.

On CI packages would be published to NPM (needs to create a token) and PYPi (needs to create a token)

Need to double check how tokens work with NPM publish.

Thoughts?

goanpeca · 2020-10-15T01:20:20Z

.bumpversion.cfg

@@ -1,20 +1,27 @@
 [bumpversion]
-current_version = 5.0.8.dev
-parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(.(?P<suffix>.+))?
+current_version = 5.0.9.dev0


Added the build part to the version, since it seems to be more standard/valid?

goanpeca · 2020-10-15T01:20:34Z

.bumpversion.cfg

-parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(.(?P<suffix>.+))?
+current_version = 5.0.9.dev0
+commit = True
+message = Bump version: {current_version} → {new_version}


We can do the commit directly

goanpeca · 2020-10-15T01:20:43Z

.bumpversion.cfg

+commit = True
+message = Bump version: {current_version} → {new_version}
+tag = False
+tag_name = v{new_version}


And add the tag directly

goanpeca · 2020-10-15T01:21:17Z

.github/workflows/release_publish.yml

+    - name: Publish NPM Package
+      env:
+        TWINE_USERNAME: __token__
+        TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}


This would need to be created on PyPI and set on the Settings secrets

goanpeca · 2020-10-15T01:21:47Z

.github/workflows/release_publish.yml

+      # Handle token for NPM?
+      # https://docs.npmjs.com/using-private-packages-in-a-ci-cd-workflow
+      run: |
+        npm publish


I think we need to create this token also, but I am not sure how it needs to be provided to the npm publish command (maybe create the npmrc file as the link above?)

goanpeca · 2020-10-19T15:44:29Z

@MSeal let me know what you think :).

MSeal · 2020-10-19T23:44:29Z

Hi Sorry @goanpeca I'm going to be fairly unresponsive for a bit as I had a new addition to the family over the weekend.

I like moving to github actions where possible and I'm not opposed to automating releases, though I've not automated any of the jupyter repos for PyPI / npm release so far since it's sometimes good to have a human take the steps explicitly. In this case I'd check with @rgbkrk, @minrk, or @takluyver in case on of them is free / has an opinion.

goanpeca · 2020-10-19T23:54:20Z

I'm going to be fairly unresponsive for a bit as I had a new addition to the family over the weekend.

Congratulations! and sure, no problem.

I have been automating this way a lot of other projects and I think it makes maintenance much easier and shareable among other team members.

This optimizes for having more maintainers, which is something we struggle to find in open source.

rgbkrk · 2020-10-21T19:49:52Z

I'm a fan of automating this. The credentials piece is usually the trickiest bit, since it needs to be encrypted in the repo but accessible by automation. What's the plan for that?

goanpeca · 2020-10-21T19:56:30Z

What's the plan for that?

Github secrets basically.

willingc · 2020-11-21T20:27:42Z

@goanpeca Hi! Do you want to finish this up and then we can cut a release with all the open PRs?

goanpeca · 2020-11-21T21:18:19Z

@goanpeca Hi! Do you want to finish this up and then we can cut a release with all the open PRs?

Hi @willingc, yes. I need to make sure the npm bits work as expected so I will try it on a personal repo with a dummy npm package.

Once I get that workin I will come an update the PR.

Cheers

goanpeca · 2020-11-29T22:19:58Z

Should we remove python 3.5 support?

goanpeca · 2020-11-29T22:27:39Z

I think this one is ready @willingc. Before we can test this, we need to add 2 Repository secrets to the Settings of this repo.

PYPI_TOKEN and NPM_TOKEN

To create the pypi one, one needs to go to https://pypi.org/manage/account/token/ and create a token for the specific package. I would suggest using a bot account (jupyter-bot) to automate this process and avoid having CI workflows attached to personal accounts.

To create the NPM one, one needs to go to https://www.npmjs.com/settings//tokens and create a new automation token. Same suggestion from before, using a bot-account is probably a better idea to handle this automation processes.

These values should be added as the secrest rspectively.

I am not currently a maintainer of this project but I would be happy to help with it and set things up, and help moving things forward :)

willingc · 2020-11-30T16:41:05Z

@goanpeca Check your inbox. I've bumped your privileges so that you can enable the secrets needed.

goanpeca · 2020-11-30T16:48:46Z

Awesome @willingc.

Now I (only) need access to https://pypi.org/project/nbformat/

Anyone in there should be able to bump me :)

And on https://www.npmjs.com/package/nbformat

I think @rgbkrk has access to both, you could add me, if you agree of course.

Cheers!

willingc · 2020-12-01T00:36:00Z

@minrk @rgbkrk Would one of you be able to add @goanpeca and me to the pypi account and the npm account? I'm willingc on both. Thanks!

rgbkrk · 2020-12-15T23:55:59Z

Will do. What are your logins for each @goanpeca?

The npm package is actually called nbformat-schema. I'll add you now @willingc.

Update: done

rgbkrk · 2020-12-16T00:00:31Z

It looks like Jupyterlab has an nbformat package that could/should be automated to receive these updates too, unless the purpose of the package is different: https://www.npmjs.com/package/@jupyterlab/nbformat

goanpeca · 2020-12-16T00:08:06Z

Hi @rgbkrk, thanks.

goanpeca on every platform :) I can ask tomorrow on the the JLab weekly meeting about @jupyterlab/nbformat to be sure.

rgbkrk · 2020-12-16T17:15:13Z

@goanpeca -- all set for npm and PyPI. I think you have to accept the invite (somehow?)

goanpeca · 2020-12-16T18:15:19Z

@rgbkrk received! Thanks :)

rgbkrk · 2020-12-16T21:56:07Z

Let me know when this is ready to merge, otherwise feel free to go forward with it @goanpeca.

goanpeca · 2020-12-16T22:05:20Z

Thanks, will look into this later today :)

goanpeca force-pushed the release-process branch 2 times, most recently from 4f39bc7 to 9e96c87 Compare October 15, 2020 00:58

goanpeca commented Oct 15, 2020

View reviewed changes

goanpeca changed the title ~~PR: Add CI workflow for pakcage release on tag push~~ PR: Add CI workflow for package release on tag push Oct 15, 2020

rgbkrk approved these changes Oct 21, 2020

View reviewed changes

willingc mentioned this pull request Nov 21, 2020

Prepare next release 5.1 #199

Closed

6 tasks

goanpeca force-pushed the release-process branch from 9e96c87 to 6ba0be3 Compare November 29, 2020 22:17

goanpeca marked this pull request as ready for review November 29, 2020 22:17

goanpeca mentioned this pull request Dec 17, 2020

Cell ID JEP Implementation #189

Merged

goanpeca force-pushed the release-process branch from fac6936 to a71334b Compare December 20, 2020 22:44

MSeal approved these changes Dec 20, 2020

View reviewed changes

goanpeca added 4 commits December 20, 2020 17:48

Add CI workflow for pakcage release on tag push

78d0565

Add scripts and handling of environment variables

e9cdc35

Clean up yaml files

41dbfc7

Update CI workflow to release with tags

a00402a

goanpeca force-pushed the release-process branch from a71334b to a00402a Compare December 20, 2020 22:49

Update MANIEST

2693e69

MSeal merged commit f404fcb into jupyter:master Dec 20, 2020

goanpeca deleted the release-process branch December 20, 2020 22:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PR: Add CI workflow for package release on tag push #195

PR: Add CI workflow for package release on tag push #195

goanpeca commented Oct 15, 2020 •

edited by rgbkrk

Loading

goanpeca commented Oct 15, 2020 •

edited

Loading

goanpeca Oct 15, 2020

goanpeca Oct 15, 2020

goanpeca Oct 15, 2020

goanpeca Oct 15, 2020

goanpeca Oct 15, 2020

goanpeca commented Oct 19, 2020

MSeal commented Oct 19, 2020

goanpeca commented Oct 19, 2020

rgbkrk commented Oct 21, 2020

goanpeca commented Oct 21, 2020

willingc commented Nov 21, 2020

goanpeca commented Nov 21, 2020

goanpeca commented Nov 29, 2020

goanpeca commented Nov 29, 2020

willingc commented Nov 30, 2020

goanpeca commented Nov 30, 2020

willingc commented Dec 1, 2020

rgbkrk commented Dec 15, 2020 •

edited

Loading

rgbkrk commented Dec 16, 2020

goanpeca commented Dec 16, 2020

rgbkrk commented Dec 16, 2020

goanpeca commented Dec 16, 2020

rgbkrk commented Dec 16, 2020

goanpeca commented Dec 16, 2020

PR: Add CI workflow for package release on tag push #195

PR: Add CI workflow for package release on tag push #195

Conversation

goanpeca commented Oct 15, 2020 • edited by rgbkrk Loading

goanpeca commented Oct 15, 2020 • edited Loading

goanpeca Oct 15, 2020

Choose a reason for hiding this comment

goanpeca Oct 15, 2020

Choose a reason for hiding this comment

goanpeca Oct 15, 2020

Choose a reason for hiding this comment

goanpeca Oct 15, 2020

Choose a reason for hiding this comment

goanpeca Oct 15, 2020

Choose a reason for hiding this comment

goanpeca commented Oct 19, 2020

MSeal commented Oct 19, 2020

goanpeca commented Oct 19, 2020

rgbkrk commented Oct 21, 2020

goanpeca commented Oct 21, 2020

willingc commented Nov 21, 2020

goanpeca commented Nov 21, 2020

goanpeca commented Nov 29, 2020

goanpeca commented Nov 29, 2020

willingc commented Nov 30, 2020

goanpeca commented Nov 30, 2020

willingc commented Dec 1, 2020

rgbkrk commented Dec 15, 2020 • edited Loading

rgbkrk commented Dec 16, 2020

goanpeca commented Dec 16, 2020

rgbkrk commented Dec 16, 2020

goanpeca commented Dec 16, 2020

rgbkrk commented Dec 16, 2020

goanpeca commented Dec 16, 2020

goanpeca commented Oct 15, 2020 •

edited by rgbkrk

Loading

goanpeca commented Oct 15, 2020 •

edited

Loading

rgbkrk commented Dec 15, 2020 •

edited

Loading